Systems, Methods and Computer Readable Storage Media Storing Instructions for Generating Planning Images Based on HDR Applicators

ABSTRACT

Systems, methods, and computer-readable storage media relate to generate an integrated image based on one or more applicators inserted into a target of a patient. The method may include receiving ultrasound image data and planning image data of the target. The ultrasound image data including a pre-operative and post-operative image data and the planning image data may include post-operative image data (e.g., CT and/or MRI image data). The method may include processing the ultrasound image data and the planning image data to determine a location of each channel and/or needle. The method may include generating an integrated image including the ultrasound image data and the planning image data based on the location of each channel and/or needle of the applicator. The methods, systems, and computer readable media according to embodiments can improve target (e.g., prostate) delineation, enable accurate dose planning and delivery, and potentially enhance HDR treatment outcome.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/700,463 filed Apr. 30, 2015, which claims priority to U.S.Provisional Application No. 61/986,410 filed Apr. 30, 2014. The entiretyof each of these applications is hereby incorporated by reference forall purposes.

BACKGROUND

Radiotherapy is an important treatment modality for prostate cancer. Thetechnological advances in real-time ultrasound image guidance forHigh-Dose-Rate (HDR) prostate brachytherapy have placed this treatmentmodality at the forefront of innovation in the field of cancerradiotherapy. HDR prostate brachytherapy generally involves using aradiation source (Iridium-192) to deliver high radiation dose to theprostate gland through a series of applicators that are temporarilyplaced within the prostate transperineally under the transrectalultrasound guidance. The dose decreases exponentially with increasingdistance from the radioactive source according to the inverse-squarelaw. This procedure allows the dose delivered to the surrounding normaltissues to be minimized.

Recent data shows an improved efficacy of this treatment approach inpatients with locally advanced cancer when compared with conventionalexternal beam radiotherapy. As a result, an increasing number of men,many of younger ages, have been undergoing prostate HDR brachytherapyinstead of radical prostatectomy for localized prostate cancer.

One key to the success of the HDR prostate brachytherapy is the accuratelocalization of the prostate, for example, in the treatment-planningimages. If the prostate is not accurately localized, a high therapeuticradiation dose could be delivered to the surrounding normal tissues(e.g. rectum and bladder) during the treatment. This may cause severecomplications such as rectum bleeding and lead to an under-treatment ofthe cancerous regions within the prostate gland; therefore result inpoor treatment outcome.

Treatment-planning images are often computer tomography (CT) images.However, CT images can have poor soft-tissue contrast and thus lead toinaccurate prostate localization and inaccurate treatment planning. Theaccuracy and reproducibility of prostate volume manually contoured on CTimages among physicians has been found to be poor. It has been shownthat CT-based prostate contours consistently overestimate the prostatevolume by 30% to 50%.

SUMMARY

Thus, there is a need for accurate prostate localization, for example,by improving the accuracy of the prostate contour. This can enableaccurate treatment planning and delivery, and therefore can improve theoutcomes from HDR treatments.

This disclosure generally relates to methods, systems, and computerreadable storage media that include instructions to integrateintra-operative ultrasound-based target volume into treatment planningthrough fusion based on the applicator locations.

In some embodiments, the methods may relate to a computer-implementedmethod to generate an integrated image based on or more applicatorsinserted into a target of a patient, each applicator including at leastone channel and/or needle. In some embodiments, the method may includereceiving ultrasound image data and planning image data of the target ofthe patient. The ultrasound image data may include a first set ofultrasound image data of the target before insertion of the one or moreapplicators and a second set of ultrasound image data of the targetafter the insertion of the one or more applicators. The planning imagedata may include image data of the target after the insertion of theplanning image data being acquired from an imaging system different fromultrasound. In some embodiments, the method may include processing theultrasound image data to determine a location of each channel and/orneedle. The processing may include registering the first set ofultrasound image data and the second set of ultrasound image data. Themethod may further include processing the planning image data todetermine a location of each channel and/or needle. The method may alsoinclude generating an integrated image including the ultrasound imagedata and the planning image data based on the location of each channeland/or needle.

In some embodiments, the computer-readable media may relate to anon-transitory computer readable storage medium comprising programinstruction stored thereon, wherein the program instructions areexecutable by a computer to cause the computer to generating anintegrated image based on one or more applicators inserted into a targetof a patient by performing steps. Each applicator may include at leastone channel and/or needle. In some embodiments, the steps may includereceiving ultrasound image data and planning image data of the target ofthe patient. The ultrasound image data may include a first set ofultrasound image data of the target before insertion of the one or moreapplicators and a second set of ultrasound image data of the targetafter the insertion of the one or more applicators. The planning imagedata may include image data of the target after the insertion of theplanning image data being acquired from an imaging system different fromultrasound. In some embodiments, the steps may include processing theultrasound image data to determine a location of each channel and/orneedle. The processing may include registering the first set ofultrasound image data and the second set of ultrasound image data. Thesteps may further include processing the planning image data todetermine a location of each channel and/or needle. The steps may alsoinclude generating an integrated image including the ultrasound imagedata and the planning image data based on the location of each channeland/or needle.

In some embodiments, the systems may relate to a system for generatingan integrated image based on or more applicators inserted into a targetof a patient, each applicator including at least one channel and/orneedle. The system may include at least one processor; and a memory. Theprocessor may be configured to cause receiving ultrasound image data andplanning image data of the target of the patient. The ultrasound imagedata may include a first set of ultrasound image data of the targetbefore insertion of the one or more applicators and a second set ofultrasound image data of the target after the insertion of the one ormore applicators. The planning image data may include image data of thetarget after the insertion of the planning image data being acquiredfrom an imaging system different from ultrasound. In some embodiments,the processor may be configured to cause processing the ultrasound imagedata to determine a location of each channel and/or needle. Theprocessing may include registering the first set of ultrasound imagedata and the second set of ultrasound image data. The processor mayfurther be configured to cause processing the planning image data todetermine a location of each channel and/or needle. The processor mayalso be configured to cause generating an integrated image including theultrasound image data and the planning image data based on the locationof each channel and/or needle.

Additional advantages of the disclosure will be series forth in part inthe description which follows, and in part will be obvious from thedescription, or may be learned by practice of the disclosure. Theadvantages of the disclosure will be realized and attained by means ofthe elements and combinations particularly pointed out in the appendedclaims. It is to be understood that both the foregoing generaldescription and the following detailed description are exemplary andexplanatory only and are not restrictive of the disclosure, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure can be better understood with the reference to thefollowing drawings and description. The components in the figures arenot necessarily to scale, emphasis being placed upon illustrating theprinciples of the disclosure.

FIG. 1 shows a block diagram illustrating a system according toembodiments;

FIG. 2 shows a block diagram illustrating an example of a computingsystem;

FIG. 3 shows a method of processing ultrasound image data and planningimage data to generate an integrated image according to embodiments;

FIG. 4 shows a method of determining locations of the body portions andtip portions of the one or more applicators according to someembodiments;

FIG. 5A shows a method of determining locations of the tip portions ofeach needle and/or channel of the applicator(s) according to otherembodiments;

FIG. 5B shows a method of modifying the tip portion of the needlesand/or channels determined in FIG. 4 based on the comparison with thetip portion determined in FIG. 5A according to some embodiments;

FIG. 6 shows a method of registering the ultrasound image data andplanning image data based on the needle and/or channel locations of theapplicator(s) according to embodiments;

FIG. 7 shows an illustrative method of processing ultrasound image dataand planning image data to generate an integrated image according toembodiments; and

FIG. 8 shows an example of results of integrating TRUS-based prostatevolume into planning images (post-operative CT images).

DETAILED DESCRIPTION OF THE EMBODIMENTS

In the following description, numerous specific details are series forthsuch as examples of specific components, devices, methods, etc., inorder to provide an understanding of embodiments of the disclosure. Itwill be apparent, however, to one skilled in the art that these specificdetails need not be employed to practice embodiments of the disclosure.In other instances, well-known materials or methods have not beendescribed in detail in order to avoid unnecessarily obscuringembodiments of the disclosure. While the disclosure is susceptible tovarious modifications and alternative forms, specific embodimentsthereof are shown by way of example in the drawings and will herein bedescribed in detail. It should be understood, however, that there is nointent to limit the disclosure to the particular forms disclosed, but onthe contrary, the disclosure is to cover all modifications, equivalents,and alternatives falling within the spirit and scope of the disclosure.

This disclosure generally relates to methods, systems, and computerreadable storage media that generate an integrated image including theultrasound image data and the planning image data based on the locationof each channel and/or needle of an HDR applicator. The methods,systems, and computer readable media according to embodiments addressthe deficiencies of relying on planning images alone.

The methods, systems, and computer readable media according toembodiments can result in improved prostate contours; result in moreefficient and faster procedure; and result in an improved registeredimage. The methods, systems, and computer readable media according toembodiments can improve prostate delineation, enable accurate doseplanning and delivery, and potentially enhance prostate HDR treatmentoutcome.

The embodiments of the disclosure are described with respect to highdose rate (HDR) brachytherapy of a prostate of a human patient. Itshould be understood by one of ordinary skill in the art that theembodiments of the disclosure can be applied to other types ofbrachytherapy and other targets (such as other portions of the body),whether human or animal. The use of the method and system of theembodiments of the disclosure can also be adapted for other types ofapplicators.

FIG. 1 shows an example of a system 100 for generating planning imagesbased on pre-operative ultrasound image data and post-operative planningimage data based on one or more HDR brachytherapy applicators. Anapplicator may be any applicator having a plurality of channels and/orneedles; a device including multiple channels and/or needles; a devicehaving a plurality of applicators and/or needles; or an organ specificdesign; among others; or a combination thereof.

In some embodiments, the modules and/or systems of the system 100 may beconnected to a data network, a wireless network, or any combinationthereof. In some embodiments, any of the modules and/or systems of thesystem 100 may be at least in part be based on cloud computingarchitecture. In some embodiments, the modules and/or systems may beapplied to a self-hosted private cloud based architecture, a dedicatedpublic cloud, a partner-hosted private cloud, as well as any cloud basedcomputing architecture. Although the modules of the system are shown asbeing directly connected, the modules may be indirectly connected to oneor more of the other modules of the system. In some embodiments, amodule may be only directly connected to one or more of the othermodules of the system.

As shown in FIG. 1, the system 100 may include an ultrasound imagingsystem 160. In some embodiments, the ultrasound (US) imaging system 160may be configured to acquire ultrasound images before and/or after theinsertion of the applicator(s). In some embodiments, the system 160 mayinclude two ultrasound systems. In some embodiments, for example, theultrasound system 160 may include a TRUS ultrasound image system.

In some embodiments, the system 100 may include a planning imagingsystem 150. In some embodiments, the planning imaging system 150 may beconfigured to acquire post-operative planning images (after insertion ofthe applicator(s)). In some embodiments, the planning image system 150may be a CT imaging system. In some embodiments, the planning system 150may be other modality, for example, may be a MM imaging system.

In some embodiments, the system 100 may include an imaging registrationsystem 110 according to embodiments. The system 110 may be configured toacquire the image data from the imaging systems (e.g., 150 and/or 160)and/or a medical imaging database 130 configured to store medicalimaging data.

The imaging registration system 110 according to embodiments may beconfigured to generate integrated post-operative images from more thanone imaging modality based on the needles and/or channels of theapplicator(s). In some embodiments, the system 110 may include a targetvolume determination module 112 configured to determine volume (orcontours) of the target (e.g., prostate) from the ultrasound image data(pre-operative). In some embodiments, the system 110 may include anapplicator detection module 114 configured to determine the locations ofthe needles and/or channels of the applicator(s) in the image data fromthe US imaging system 160 and the planning image system 150.

In some embodiments, the system 100 may include an image registrationmodule 116 configured to register the image data from the planning imagesystem 150 and the US imaging system 160 based on the determinedlocations of the needles and/or channels of the applicator(s). In someembodiments, the image integration module 118 may be configured tointegrate the target volume into the registered image data.

In some embodiments, the system may include an HDR treatment planningsystem 140. The HDR treatment planning system may be configured todeliver HDR brachytherapy treatment based on the integrated imagesgenerated by the imaging registration system 110.

One or more of the modules and/or systems of system 100 may be and/orinclude a computer system and/or device. FIG. 2 is a block diagramshowing a computer system 200. The modules of the computer system 200may be included in at least some of the systems and/or modules, as wellas other devices of system 100.

The systems may include any number of modules that communicate withother through electrical or data connections (not shown). In someembodiments, the modules may be connected via a wired network, wirelessnetwork, or combination thereof. In some embodiments, the networks maybe encrypted. In some embodiments, the wired network may be, but is notlimited to, a local area network, such as Ethernet, or wide areanetwork. In some embodiments, the wireless network may be, but is notlimited to, any one of a wireless wide area network, a wireless localarea network, a Bluetooth network, a radiofrequency network, or anothersimilarly functioning wireless network.

It is also to be understood that the systems may omit any of the modulesillustrated and/or may include additional modules not shown. It is alsobe understood that more than one module may be part of the systemalthough one of each module is illustrated in the system. It is furtherto be understood that each of the plurality of modules may be differentor may be the same. It is also to be understood that the modules mayomit any of the components illustrated and/or may include additionalcomponent(s) not shown.

In some embodiments, the modules provided within the systems may be timesynchronized. In further embodiments, the systems may be timesynchronized with other systems, such as those systems that may be onthe medical facility network.

The system 200 may be a computing system, such as a workstation,computer, or the like. The system 200 may include one or more processors212. The processor(s) 212 (also referred to as central processing units,or CPUs) may be any known central processing unit, a processor, or amicroprocessor. The processor(s) 212 may be coupled directly orindirectly to one or more computer-readable storage media (e.g., memory)214. The memory 214 may include random access memory (RAM), read onlymemory (ROM), disk drive, tape drive, etc., or a combinations thereof.The memory 214 may be configured to store programs and data, includingdata structures. In some embodiments, the memory 214 may also include aframe buffer for storing data arrays.

The processor(s) 212 may be configured to determine location(s) of eachneedle and/or channel and generate an integrated image. In someembodiments, the processor(s) 212 may be capable of performing the imagedata processing. In other embodiments, the system may include a separateprocessor(s) (e.g., CPU) for performing the image data processing and/orgeneration of integrated image.

In some embodiments, another computer system may assume the dataanalysis or other functions of the processor(s) 212. In response tocommands received from the input device, the programs or data stored inthe memory 214 may be archived in long term storage or may be furtherprocessed by the processor and presented on a display.

In some embodiments, the system 210 may include a communicationinterface 218 configured to conduct receiving and transmitting of databetween other modules on the system and/or network. The communicationinterface 218 may be a wired and/or wireless interface, a switchedcircuit wireless interface, a network of data processing devices, suchas LAN, WAN, the internet, or combination thereof. The communicationinterface may be configured to execute various communication protocols,such as Bluetooth, wireless, and Ethernet, in order to establish andmaintain communication with at least another module on the network.

In some embodiments, the system 210 may include an input/outputinterface 216 configured for receiving information from one or moreinput devices 230 (e.g., a keyboard, a mouse, and the like) and/orconveying information to one or more output devices 240 (e.g., aprinter, a CD writer, a DVD writer, portable flash memory, etc.). Insome embodiments, the one or more input devices 230 may configured tocontrol, for example, the determination of the needle and/or channellocations and/or integrated image, display of the reconstructedneedle(s) and/or channel(s) and/or integrated image on a display 250,printing of the images by a printer interface, among other things.

FIG. 3 illustrates a method 300 for generating integrated planningimages based on one or more brachytherapy applicators according toembodiments. The system for carrying out the embodiments of the methodsdisclosed herein is not limited to the systems shown in FIGS. 1 and 2.Other systems may be used.

The methods of the disclosure are not limited to the steps describedherein. The steps may be individually modified or omitted, as well asadditional steps may be added. It will be also understood that at leastsome of the steps may be performed in parallel.

Unless stated otherwise as apparent from the following discussion, itwill be appreciated that terms such as “identifying,” “receiving,”“integrating,” “filtering,” “combining,” “reconstructing,” “segmenting,”“generating,” “registering,” “determining,” “obtaining,” “processing,”“computing,” “selecting,” “estimating,” “detecting,” “tracking,”“calculating,” “comparing,” “modifying,” “aligning” “fusing,” or thelike may refer to the actions and processes of a computer system, orsimilar electronic computing device, that manipulates and transformsdata represented as physical (e.g., electronic) quantities within thecomputer system's registers and memories into other data similarlyrepresented as physical quantities within the computer system memoriesor registers or other such information storage, transmission or displaydevices. Embodiments of the methods described herein may be implementedusing computer software. If written in a programming language conformingto a recognized standard, sequences of instructions designed toimplement the methods may be compiled for execution on a variety ofhardware platforms and for interface to a variety of operating systems.In addition, embodiments are not described with reference to anyparticular programming language. It will be appreciated that a varietyof programming languages may be used to implement embodiments of thedisclosure.

In some embodiments, the method 300 may include a step 310 of receiving(also referred to as “first”) ultrasound image data. The ultrasoundimage data may include pre-operative (pre-applicator insertion) imagedata and post-operative (post-applicator insertion) image data. In someembodiments, the pre-operative image data may be acquired before theinsertion of the brachytherapy applicator(s) into a target using anultrasound system. The post-operative image data may be acquired afterthe insertion of the brachytherapy applicator(s) into the target butbefore the application of the high dose rate source into theapplicator(s). In some embodiments, the image data may be received fromthe image acquisition device (e.g., 3D TRUS system) (e.g., US imagingsystem 160) and/or from a data storage device (e.g., the medical imagingdatabase 130).

In some embodiments, the method 300 may include a step 320 of receiving(also referred to as “second image data”) planning image data. Theplanning image data may be received from a data storage device (e.g.,the medical imaging database 130) or the image acquisition device (e.g.,the planning imaging system 150). The planning image data may be from adifferent imaging modality (than the data received in step 310). In someembodiments, the planning image data may be acquired by a CT systemafter the applicator insertion. In some embodiments, the planning imagedata may correspond to imaging modality data different from ultrasoundacquired after the applicator insertion. In some embodiments, theplanning image data may correspond to MRI image data acquired after theapplicator insertion.

The image data received in steps 310 and/or 320 may be in a DigitalImaging and Communications in Medicine (DICOM) format. The image datamay include header and image data. The header may include imageinformation. The image information may include information regarding thescan. The image information may include but is not limited to pixelcounts, acquisition angles, distance between the patient and thedetector, distance between the source and the detector, number offrames, dimensions of the image, data resolution, and image size. Theimage data may be raw data or processed image data.

In some embodiments, the method 300 may include a step 330 ofdetermining the target contours (e.g., prostate) using the ultrasoundimage data. The step 330 may be performed using any methods to contourthe target (e.g., prostate) volume.

In some embodiments, the method 300 may include a step 340 ofdetermining location(s) (with respect to the image (e.g., pixellocation(s)) of the needles and/or channels of the applicator(s) in theultrasound image data by reconstructing the needles and/or channels fromthe ultrasound image data and a step 342 of reconstructing the needlesand/or channels from the planning image data. The steps 340 and 342 maybe performed in parallel, sequentially, or a combination thereof. Insome embodiments, a different method may be used.

In some embodiments, the location(s) of the needles and/or channels inthe ultrasound image data may be determined by reconstructing the whole(e.g., body portion and tip portion) needle(s) and/or channel(s) and byreconstructing the tip portion of each needle and/or channel. The tipportion of each needle and/or channel corresponds to the startingposition (also referred to as “tip position”) of each needle and/orchannel. The tip portion of the needles and/or channels refers to regionof the needles and/or channels located closest to the base of the target(e.g., prostate)(and farthest away from the clinician) and the bodyportion of the needles and/or channels refers to region of the needlesand/or channels that is visible in the image and that extends betweenthe tip/starting position and base of the applicator (e.g., fixationelement) that is farthest away from the base of the target (and closestto the clinician). By separately determining the tip position, thedifferent geometries of applicators can be addressed thereby improvingreconstruction of the applicator. By way of example, many applicatorshave bodies (body of the whole needle and/or channel) are cylindricaland tips that are cylindrical. The cylindrical tips can be affected byartifacts in ultrasound images and therefore can be difficult todetermine using conventional methods.

FIG. 4 show an example of a method for determining the location(s) ofthe whole channel and/or needle of applicator from the ultrasound imagedata. FIG. 5 shows an example of the method of determining the startingposition (e.g., tip) of each needle and/or channel from the ultrasoundimage data. In some embodiments, the starting position of each needleand/or channel may be first determined, for example, as shown in FIG. 5.The starting positions determined in FIG. 5 may be used as startingpoints to reconstruct the respective needle and/or channel according tothe method of FIG. 4. In other embodiments, each needle and/or channellocation may be reconstructing according to FIG. 4 and the startingpositions of each needle and/or channel may be modified based on thestarting positions determined in FIG. 5. In some embodiments, thereconstruction of the applicator(s) may be performed by other methods.

FIG. 4 shows a method 400 of determining the locations of the wholeneedles and/or channels by reconstructing the body portion and tipportion of the applicator(s) based on the ultrasound image dataaccording to embodiments. In some embodiments, the method 400 mayinclude a step 430 of registering pre-operative ultrasound data 410 andpost-operative ultrasound data 420 using an intensity and feature-basedregistration method. For example, the registration may be achieved usinga combination of an intensity-based similarity metric (such as, e.g.,normalized mutual information) and a feature-based similarity metric(such as, e.g., normalized sum-of-squared differences metric).

Next, the method 400 may include a step 440 processing the ultrasoundpre-operative data 410 to generate an attenuation correction map (e.g.,a 3D correction map). The attenuation correction map may be generated byany method. In some embodiments, the attenuation correction map may begenerated by applying an inhomogeneous correction filter to thepre-operative ultrasound image to generate a patient-specificattenuation correction map. In some embodiments, other methods may beused to generate attenuation correction map. For example, theattenuation correction map may be generated by universally applying apredetermined attenuation coefficient (e.g., 0.5 dB/MHZ/cm) to thepre-operative ultrasound image. This attenuation correction map cancorrect for intensity changes induced by various machine settings (e.g.,time-gain-control) and tissue attenuation (ultrasound signal decreaseswhile traveling though the tissue).

The method 400 may include a step 450 of transforming the correction mapto the post-operative ultrasound data 420. Next, the method 400 mayinclude a step 460 of applying the transformed correction map to thepost-operative image data 420. The step 460 may include multiplying thetransformed 3D correction map to the post-operative image data 420 togenerate a corrected post-operative ultrasound images.

In some embodiments, the method 400 may include a step of detecting theneedles and/or channels of the applicator(s) in the correctedpost-operative ultrasound images by filtering and thresholding theintensity values, for example, using a predetermined range of one ormore threshold values. The threshold(s) for the intensity values maydepend on one or more factors, including but not limited to scanparameters, applicator materials, patient soft-tissue variations, amongothers, or a combination thereof. For example, for an 8-bit ultrasoundimage (intensity level 0-255), the threshold value may be 220.

FIG. 5A shows a method 500 of determining the starting position (e.g.,location of the tip portion) of each needle and/or channel of theapplicator(s) in the ultrasound image data 310 by reconstructing basedsolely on the post-operative and attenuation-corrected ultrasound imagedata according to some embodiments. The method 500 may include a step510 of determining the starting position (e.g., tip) of each needleand/or channel located close to the base of the target (e.g., prostate).In some embodiments, the starting position may be determined bydetermining the end point of the brightest point. Next, the method 500may include a step 520 of determining an area of the highest intensity(e.g., the brightest point) of each needle and/or channel andidentifying that region, for example, by placing a circle with theapplicator diameter on the image (e.g., an axial TRUS image). In someembodiments, steps 510 and 520 may repeated for each image. In otherembodiments, steps 510 and 520 may be applied to a portion of the imagesand the location of the starting position (tip) of each needle and/orchannel in the remaining images may be determined by interpolation. Insome embodiments, the method 500 may include a step 530 ofreconstructing the starting positions of the needles and/or channels ofthe applicator(s) for the ultrasound images based on the identifiedregions. In some embodiments, the reconstruction may be 3D.

In some embodiments, the step 340 may further include steps to modifythe starting position(s) of the needles and/or channels determined inFIG. 4 with the starting position(s) of the needles and/or channelsdetermined in FIG. 5A. FIG. 5B shows an example of a method 550 ofmodifying the starting position(s) of the needles and/or channelsdetermined in FIG. 4 based on the comparison according to someembodiments. For example, the method 550 may include a step 560 ofcomparing the starting position(s) of the whole needles and/or channelsdetermined in FIG. 4 with the respective starting position(s) determinedin FIG. 5A to determine whether they match. If they match (Yes at step570), then no further processing is necessary (step 580) and the needlelocations determined in FIG. 4 may be outputted in step 340 for furtherprocessing by the method 300. If it is determined that the startingposition(s) do not match, then the locations determined in FIG. 4 may befurther processed based on the difference between the startingpositions. For example, if it is determined that the starting positionof the whole channel(s) and/or needle(s) determined in FIG. 4 is shorterthan the respective starting (tip) position in FIG. 5A (e.g., becausethere is a position gap between the tip portion of the needle and/orchannel determined in FIG. 4 and the tip portion determined in FIG. 5A),the method 550 may further include a step 572 of interpolating(extending) the whole needle (e.g., body portion and the tip portion)determined in FIG. 4 till the starting (tip) position of FIG. 4 matchesthe starting (tip) position determined in FIG. 5A. This can resolve themissing part of the needle and/or channel due to artifacts in theultrasound images that may have occurred when the needle and/or channellocations are determined in the method of FIG. 4. If the startingposition of a whole needle and/or channel detected in the method of FIG.4 is determined to be longer (e.g., extend past) the starting (tip)position determined in the method of FIG. 5A, the step 550 may include astep 574 of removing or deleting that part of the needle and/or channeldetermined in FIG. 4 so that it matches with the respective starting(tip) position determined in FIG. 5A. This can resolve anyover-detection due to artifacts in ultrasound images that may haveoccurred when the needle/channel locations are determined according themethod of FIG. 4. The location of the needles and/or channels determinedin the steps 572 and/or 574 may be outputted in step 340 for furtherprocessing by the method 300.

In step 342, for reconstructing the needles and/or channels of theapplicator(s) in the planning images (e.g., CT images), the locations ofthe needles and/or channels of the applicators may be determined in theplanning images (e.g., CT) by reconstruction. In some embodiments, thestep may include filtering the intensity using a threshold to detect theneedles and/or channels. This threshold may be based on the applicatordiameter. In some embodiments, the threshold may be about 950 HU. Insome embodiments, other methods may be used.

In some embodiments, the method 300 may include a step 350 ofregistering the locations of the needles and/or channels of theapplicator(s) between the ultrasound images determined in step 340 andthe planning images determined in step 342 to determine a transformationfield to transform target (e.g. prostate) contour in ultrasound imagesto planning (e.g. CT) images. In this way, the needles and/or channelscan serve as landmark to fuse the US and planning images.

In some embodiments, the ultrasound images and the planning images maybe registered according to the method shown in FIG. 6. FIG. 6 shows amethod 600 of registering the US image and the planning images. In otherembodiments, other registration methods may be used.

In some embodiments, the method 600 may include a step 610 of samplingpoints from the needles and/or channels of the applicator(s) determinedfrom the post-operative ultrasound image data and a step 620 of samplingpoints from the needles and/or channels of the applicator(s) from theplanning image data based on the locations of the needles and/orchannels determined in steps 340 and 342, respectively. In someembodiments, the points may be sampled using a local distinctivenessmeasure. In other words, less sample points can be required in areas ofsmall changes (smooth areas), while more sample points can be requiredin areas of greater changes. Please see, e.g., Alexa M, Behr J, Cohen-OrD, Fleishman S, Levin D, Silva C T. Computing and rendering point setsurfaces. Visualization and Computer Graphics, IEEE Transactions on.2003; 9(1):3-15. doi: 10.1109/tvcg.2003.1175093; and Torsello A, Rodola,x, E., Albarelli A, editors. Sampling Relevant Points for SurfaceRegistration. 3D Imaging, Modeling, Processing, Visualization andTransmission (3DIMPVT), 2011 International Conference on; 2011 16-19 May2011. Next, the method 600 may include a step 630 of registering theimages by matching the points from the ultrasound images and theplanning images. In some embodiments, the points may be matched using afuzzy-to-deterministic method. In some embodiments, the correspondencesbetween the two point sets may be described by fuzzy correspondencematrixes. The similarity between two sets of applicator landmarks in US(e.g., TRUS images) and planning images (e.g., CT images) can be definedby a Euclidean distance between their point sets. A soft assigntechnique may then be used to optimize the fuzzy correspondence matrixesto minimize the Euclidean distance between their point sets. In someembodiments, other methods may be used.

The method 300 may include a step 360 of integrating the US-based targetcontours determined in step 330 and the planning images received in step320 to generate integrated planning images. In some embodiments, thestep 360 may include transforming the US-based target contours (e.g.,volume determined in step 330) using the transformation field(determined in step 350) from US-planning image registration. In someembodiments, the transformation may be B-Spline spatial transformation.In other embodiments, the transformation may be a differenttransformation method.

In some embodiments, the method may further include a step 370 ofoutputting the generated integrated image(s). In some embodiments, theoutputting may include displaying, printing, storing, and/ortransmitting the generated image(s). In some embodiments, the integratedimage(s) may be transmitted to another system, server and/or storagedevice for the printing, displaying and/or storing the generated images.

In some embodiments, the method may further include transmitting thegenerated image(s) to another system. In some embodiments, the methodmay further include transmitting the generated images to a treatmentplanning system, such as HDR treatment planning system 140.

FIG. 7 shows an illustrative overview of the method integratingTRUS-based prostate volume into planning images based on the needleand/or channels of the applicator(s). FIG. 8 shows an example of resultsof integrating TRUS-based prostate volume into planning images(post-operative CT images). In FIG. 8, a1-a3 are TRUS images in axial,coronal and sagittal directions; b1-b3 are the post-operative CT; c1-c3are the post-registration TRUS images; and d1-d3 are the TRUS-CT fusionimages, where the prostate volume is integrated.

In some embodiments, the steps of the methods may be performed over awired, wireless, or combination thereof. In some embodiments, thenetworks may be encrypted. In some embodiments, the wired network maybe, but is not limited to, a local area network, such as Ethernet, orwide area network. In some embodiments, the wireless network may be, butis not limited to, any one of a wireless wide area network, a wirelesslocal area network, a Bluetooth network, a radio frequency network, oranother similarly functioning wireless network.

While the disclosure has been described in detail with reference toexemplary embodiments, those skilled in the art will appreciate thatvarious modifications and substitutions may be made thereto withoutdeparting from the spirit and scope of the disclosure as series forth inthe appended claims. For example, elements and/or features of differentexemplary embodiments may be combined with each other and/or substitutedfor each other within the scope of this disclosure and appended claims.

What is claimed is:
 1. A computer-implemented method for generating anintegrated image based on one or more applicators inserted into a targetof a patient, comprising: receiving ultrasound image data and planningimage data of the target of the patient, the ultrasound image dataincluding (i) a first set of ultrasound image data of the target beforeinsertion of one or more applicators that includes at least one channeland/or needle, each channel and/or needle including a tip portion andbody portion; and (ii) a second set of ultrasound image data of thetarget after the insertion of the one or more applicators and beforeapplication of one or more radiation sources into the one or moreapplicators, the planning image data including image data of the targetafter the insertion of the planning image data being acquired from animaging system different from ultrasound; processing the ultrasoundimage data to determine a location of a body portion and/or a tipportion of each channel and/or needle, the processing includingregistering the first set of ultrasound image data and the second set ofultrasound image data; determining a location of body portion and/or tipportion of each channel and/or needle from the planning image data;registering the ultrasound image data and the planning image data usingone or more sampling points from the at least one channel and/or needlefrom the second set of ultrasound image data and the planning image databased on the location of each channel and/or needle; and generating anintegrated image of the target including the ultrasound image data andthe planning image data based on the location of each channel and/orneedle of the one or more applicators.
 2. The method according to claim1, further comprising: processing the first set of ultrasound image datato generate a correction map; and applying the correction map to thesecond set of ultrasound image data to determine the location of eachchannel and/or needle of the one or more applicators.
 3. The methodaccording to claim 1, further comprising: sampling a plurality of pointsfrom each channel and/or needle from the second set of ultrasound imagedata and the planning image data based on the location of each channeland/or needle.
 4. The method according to claim 1, wherein the planningimage data is acquired by a CT imaging system.
 5. The method accordingto claim 1, wherein the processing the ultrasound image data todetermine the location of a body portion and/or the tip portion of eachchannel and/or needle includes: generating an attenuation correction mapbased on the first set of ultrasound image data; transforming theattenuation correction map to second set of ultrasound image data togenerate a corrected second set of ultrasound image data; anddetermining the location of each channel and/or needle from thecorrected second set of ultrasound image data.
 6. The method accordingto claim 5, wherein the determining the location of the tip portion ofeach needle and/or channel is based on the corrected second set ofultrasound image data.
 7. The method according to claim 1, wherein thetarget is a prostate of the patient and the applicator is ahigh-dose-rate prostate brachytherapy applicator.
 8. A non-transitorycomputer readable storage medium comprising program instruction storedthereon, wherein the program instructions are executable by a computerto cause the computer to generating an integrated image based on one ormore applicators inserted into a target of a patient, by performingsteps comprising: receiving ultrasound image data and planning imagedata of the target of the patient, the ultrasound image data including(i) a first set of ultrasound image data of the target before insertionof one or more applicators that includes at least one channel and/orneedle, each channel and/or needle including a tip portion and bodyportion; and (ii) a second set of ultrasound image data of the targetafter the insertion of the one or more applicators and beforeapplication of one or more radiation sources into the one or moreapplicators, the planning image data including image data of the targetafter the insertion of the planning image data being acquired from animaging system different from ultrasound; processing the ultrasoundimage data to determine a location of a body portion and/or a tipportion of each channel and/or needle, the processing includingregistering the first set of ultrasound image data and the second set ofultrasound image data; determining a location of body portion and/or tipportion of each channel and/or needle from the planning image data;registering the ultrasound image data and the planning image data usingone or more sampling points from the at least one channel and/or needlefrom the second set of ultrasound image data and the planning image databased on the location of each channel and/or needle; and generating anintegrated image of the target including the ultrasound image data andthe planning image data based on the location of each channel and/orneedle of the one or more applicators.
 9. The non-transitory computerreadable storage medium according to claim 8, further comprising programinstructions that, when executed by the computer, cause the computer toperform steps comprising: processing the first set of ultrasound imagedata to generate a correction map; and applying the correction map tothe second set of ultrasound image data to determine the location ofeach channel and/or needle of the one or more applicators.
 10. Thenon-transitory computer readable storage medium according to claim 8,further comprising program instructions that, when executed by thecomputer, cause the computer to perform steps comprising: sampling aplurality of points from each channel and/or needle from the second setof ultrasound image data and the planning image data based on thelocation of each channel and/or needle.
 11. The non-transitory computerreadable storage medium according to claim 8, wherein the planning imagedata is acquired by a CT imaging system.
 12. The non-transitory computerreadable storage medium according to claim 8, wherein the processing theultrasound image data to determine the location of the body portionand/or the tip portion of each channel and/or needle includes:generating an attenuation correction map based on the first set ofultrasound image data; transforming the attenuation correction map tosecond set of ultrasound image data to generate a corrected second setof ultrasound image data; and determining the location of each channeland/or needle from the corrected second set of ultrasound image data.13. The non-transitory computer readable storage medium according toclaim 8, wherein the determining the location of the body portion andthe tip portion of each needle and/or channel based on the first set ofultrasound image data and the second set of ultrasound image dataincludes: registering the first set of ultrasound image data and thesecond set of ultrasound image data; generating an attenuationcorrection map based on the first set of ultrasound image data;transforming the attenuation correction map to second set of ultrasoundimage data to generate corrected second set of ultrasound image data;and determining the location of each channel and/or needle from thecorrected second set of ultrasound image data.
 14. The non-transitorycomputer readable storage medium according to claim 13, wherein thedetermining the location of the tip portion of each needle and/orchannel is based on the corrected second set of ultrasound image data.15. The non-transitory computer readable storage medium according toclaim 8, wherein the target is the prostate and the applicator is ahigh-dose-rate prostate brachytherapy applicator.
 16. A system forgenerating an integrated image based on one or more applicators insertedinto a target of a patient, the system comprising: at least oneprocessor; and a memory, wherein the processor is configured to cause:receiving ultrasound image data and planning image data of the target ofthe patient, the ultrasound image data including (i) a first set ofultrasound image data of the target before insertion of one or moreapplicators that includes at least one channel and/or needle, eachchannel and/or needle including a tip portion and body portion; and (ii)a second set of ultrasound image data of the target after the insertionof the one or more applicators and before application of one or moreradiation sources into the one or more applicators, the planning imagedata including image data of the target after the insertion of theplanning image data being acquired from an imaging system different fromultrasound; processing the ultrasound image data to determine a locationof a body portion and/or a tip portion of each channel and/or needle,the processing including registering the first set of ultrasound imagedata and the second set of ultrasound image data; determining a locationof body portion and/or tip portion of each channel and/or needle fromthe planning image data; registering the ultrasound image data and theplanning image data using one or more sampling points from the at leastone channel and/or needle from the second set of ultrasound image dataand the planning image data based on the location of each channel and/orneedle; and generating an integrated image of the target including theultrasound image data and the planning image data based on the locationof each channel and/or needle of the one or more applicators.
 17. Thesystem according to claim 16, wherein the processing the ultrasoundimage data to determine a location of a body portion and/or a tipportion of each channel and/or needle includes: generating anattenuation correction map based on the first set of ultrasound imagedata; transforming the attenuation correction map to second set ofultrasound image data to generate a corrected second set of ultrasoundimage data; and determining the location of each channel and/or needlefrom the corrected second set of ultrasound image data, wherein thedetermining the location of the tip portion of each needle and/orchannel is based on the corrected second set of ultrasound image data.18. The system according to claim 16, wherein the generating includes:sampling a plurality of points from each channel and/or needle from thesecond set of ultrasound image data and the planning image data based onthe location of each channel and/or needle.
 19. The system according toclaim 16, wherein the determining the location of the body portion andthe tip portion of each needle and/or channel based on the first set ofultrasound image data and the second set of ultrasound image dataincludes: registering the first set of ultrasound image data and thesecond set of ultrasound image data; generating an attenuationcorrection map based on the first set of ultrasound image data;transforming the attenuation correction map to second set of ultrasoundimage data to generate corrected second set of ultrasound image data;and determining the location of each channel and/or needle from thecorrected second set of ultrasound image data.
 20. The system accordingto claim 16, wherein the planning image data is acquired by a CT imagingsystem, wherein the target is a prostate of the patient and theapplicator is a high-dose-rate prostate brachytherapy applicator.